Methods and systems for determining part wear based on digital image of part

ABSTRACT

Systems and methods are disclosed for determining part wear using a mobile device. One such exemplary method includes capturing, using the mobile device, at least one digital image of a wear part of a machine. The method further includes determining, by the mobile device and based on the at least one digital image, a degree of wear of the wear part.

PRIORITY CLAIM

This application is a divisional of U.S. patent application Ser. No.14/055,663, filed Oct. 16, 2013, which claims priority to U.S.Provisional Application No. 61/715,085, filed Oct. 17, 2012 and entitled“Methods and Systems for Determining Part Wear Based on Digital Image ofPart,” the entire contents for all of which are incorporated herein byreference in their entireties.

TECHNICAL FIELD

The disclosure relates generally to determining part wear and, moreparticularly, to methods and systems for determining part wear using atleast one digital image of the part.

BACKGROUND

Machines are made up of various parts. Some machine parts wear over timeas the machine is used and eventually need to be replaced. For example,a track-type tractor has track links and other parts that are subject toforces as the tractor moves material on a worksite, wearing the partsover time.

It is desirable to know the degree of wear of a part, for example, sothat its remaining useful life can be determined or estimated. But itcan be difficult to do so without specific knowledge or informationpertaining to the part or the machine.

SUMMARY

One aspect of the disclosure relates to a method for determining partwear using a mobile device. In one embodiment, the method may includecapturing, using the mobile device, at least one digital image of a wearpart of a machine. The method may further include determining, by themobile device and based on the at least one digital image, a degree ofwear of the wear part.

Another aspect of the disclosure relates to a mobile device configuredfor determining part wear. In one embodiment, the mobile device mayinclude a camera, an input device for receiving input from a user of themobile device, a storage device storing an application, and a processor.The processor may be configured to execute the stored application toreceive, via the input device, a command from the user to capture animage, and capture, using the camera and responsive to the command, atleast one digital image of a wear part of a machine. The processor maybe further configured to determine, based on the at least one digitalimage, a degree of wear of the wear part.

Still another aspect of the disclosure relates to a method fordetermining part wear using a mobile device. In one embodiment, themethod may include receiving, over an electronic communication networkfrom the mobile device, at least one digital image of a wear part of amachine. The method may further include determining, based on the atleast one digital image, a degree of wear of the wear part, and sending,over the electronic communication network to the mobile device, anindication of the determined degree of wear of the wear part.

Yet another aspect of the disclosure relates to a system for determiningpart wear using a mobile device. In one embodiment, the system includesa network communication interface configured to communicate over anelectronic communication network, a storage device storing anapplication, and a processor. The processor may be configured to executethe stored application to receive, over the electronic communicationnetwork from the mobile device, at least one digital image of a wearpart of a machine, and to determine, based on the at least one digitalimage, a degree of wear of the wear part. Additionally, the processormay be configured to send, over the electronic communication network tothe mobile device, an indication of the determined degree of wear of thewear part.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a representation of an exemplary environment for determiningpart wear, consistent with the disclosed embodiments;

FIG. 2 is a representation of an exemplary mobile device of theenvironment;

FIG. 3 is a representation of an exemplary wear part—a track link;

FIG. 4 is a representation of an exemplary parts image processing systemof one embodiment of the environment consistent with the disclosure;

FIG. 5 is a representation of an exemplary graphical user interface(GUI) display of the mobile device, consistent with the disclosedembodiments;

FIG. 6 is a flowchart of an exemplary method for determining the degreeof wear of a wear part from a digital image, consistent with thedisclosed embodiments;

FIG. 7 is a flowchart of an exemplary step of the method—calibrating acaptured digital image of a wear part—consistent with the disclosedembodiments;

FIG. 8 is a flowchart of another exemplary step of the method—measuringa wear edge of a wear part from the digital image—consistent with thedisclosed embodiments; and

FIG. 9 is a flowchart of another exemplary method for determining thedegree of wear of a wear part from a digital image, consistent with thedisclosed embodiments.

FIG. 10 is another representation of an exemplary wear part—a tracklink;

FIG. 11 is a flowchart describing an alternative method of determiningthe degree of wear of a wear part from a digital image, consistent withthe disclosed embodiments; and

FIG. 12 is a representation of an exemplary image-capturing process,consistent with the disclosed embodiments.

DETAILED DESCRIPTION

Reference will now be made in detail to exemplary embodiments, examplesof which are illustrated in the accompanying figures. The followingdescription refers to the accompanying drawings in which the samenumbers in different drawings represent the same or similar elementsunless otherwise represented. The implementations set forth in thefollowing description of exemplary embodiments do not represent allimplementations consistent with the disclosure. Instead, they are merelyexamples of systems and methods consistent with aspects of thedisclosure as recited in the appended claims.

FIG. 1 illustrates an exemplary environment 100 for determining partwear based on a digital image of the part, consistent with the disclosedembodiments. As explained below, elements of environment 100 mayinteract to enable a user to easily determine the degree of wear of apart of a machine, simply by capturing a digital image of the part usinga mobile device. As illustrated, environment 100 may include a machinesite 102. Depending upon the configuration, environment 100 mayadditionally include a parts image processing system 104, an applicationstore 106, and/or a dealer system 108. Elements of environment 100 maybe connected to an electronic communication network 109 over which theymay communicate with one another.

At machine site 102, there may be a machine 110 as well as a user 112carrying a mobile device 114. Machine site 102 represents any locationat which machine 110 and user 112 may be present. For example, machinesite 102 may be a worksite, a repair shop, a dealership, an owner'sresidence, a highway or road, etc. As explained in detail below, atmachine site 102, user 112 may capture a digital image of a wear part ofmachine 110 using mobile device 114 to determine a degree of wear of thepart.

Machine 110 may be one of any variety made up of individual parts.Certain parts of machine 110, more so than others, are subject to forcesduring operation of machine 110 that cause them to wear over time and toneed replacement. For example, machine 110 may represent a work machine,such as a track-type tractor, a wheel loader, a generator set, an oildrill, or any other type of machine that performs an intensive worktask. As shown in FIG. 1, a tractor, for instance, might havehigh-stress parts like tracks 116 made up of individual track links, ablade 118 with an edge component for moving material, and/or other partsthat wear over time as the tractor goes about its various tasks. Machine110 may also embody an industrial transport machine, such as alocomotive, a haul truck, a bus, an aircraft, or another such machinethat moves people or payloads. An aircraft, for example, might haveturbine fan blades, bleed ports, or other parts subject to high stressesthat cause them to wear over time and need periodic replacement. Machine110 may also embody a vehicle, such as a passenger truck or car. Suchmachines also have high-stress parts, such as axles or tires, that wearwith use and eventually need replacing. Machine 110 may even embody awork tool, such as a saw or drill, having one or more parts, such asteeth or bits, which wear over time with use. This description sometimesrefers to such parts of a machine 110 that are subject to stresses thatcause to wear over time with use as “wear parts.”

User 112 represents any person concerned with machine 110. For example,user 112 may be the owner or operator of machine 110. User 112 may alsobe a technician, repairperson, customer service representative, dealerpersonnel, or other serviceperson associated with machine 110. Theparticular nature of user 112 is not critical to the disclosedembodiments—user 112 may be any person who uses the disclosed systemsand methods to determine the degree of wear of a wear part of machine110. As mentioned above and explained in more detail below, user 112 maycarry a mobile device 114 and use it to capture a digital image of awear part of machine 110, which is used to determine its degree of wear.

Parts image processing system 104 may represent a computing systemassociated with an entity that makes available to user 112 the disclosedservice for determining part wear as well as other related services.That entity, for instance, might be a dealer that sells machine 110 touser 112, a lessor that leases machine 110 to user 112, a manufacturerof parts for machine 110, or a seller of parts for machine 110. In otherembodiments, that entity may be an insurance provider for machine 110 oruser 112, a warranty servicer for machine 110, a lien holder to machine110, or another third party having some relationship to machine 110 oruser 112. As explained below in more detail, parts image processingsystem 104 may have any number or combination of computing elementsenabling it to communicate, store, and process data to carry out thedisclosed techniques. For example, parts image processing system 104 mayembody a server computer or a collection of server computers configuredto perform the described techniques.

Generally, parts image processing system 104 may interact andcommunicate with other elements of environment 100, such as mobiledevice 114 and dealer system 108, to process a captured digital image ofa wear part of machine 110 and determine its wear. Depending upon theembodiment, parts image processing system 104 may also perform otherparts-related services, such as notifying dealer system 108 when it isdetermined that a part of machine 110 is sufficiently worn, so that thedealer may take action if warranted. These and other characteristics ofparts image processing system 104 are discussed in detail below.

Depending upon the configuration of environment 100, parts imageprocessing system 104 may have different roles or different degrees ofinvolvement in carrying out the disclosed techniques. For example,environment 100 may be configured as a “server-based” environment or a“cloud” environment that performs the disclosed part-wear-determinationtechniques as part of a service over network 109. In such a server orcloud environment, parts image processing system 104 (i.e., the serveror “cloud”), for example, may receive digital images of wear parts frommobile device 114 over network 109. Parts image processing system 104may then process the images to determine the degree of wear of theparts, and return results of the processing to mobile device 114 overnetwork 109. Thus, in a server or cloud environment, parts imageprocessing system 104 may do the “heavy lifting” in terms of computing,while mobile device 114 may operate as a lightweight portal (e.g.,application or browser) that allows user 112 to access the services ofparts image processing system 104 over network 109. Alternatively,environment 100 may be configured as a “client-side” environment inwhich mobile device 114 performs the bulk of the processing locally. Insuch configurations, parts image processing system 104 may have areduced role or may even be omitted entirely. This descriptioncontemplates both configurations for environment 100 and providesspecific examples of each below.

Application store 106 may represent a computing system of an entity thatmakes available software applications (e.g., “apps”) to mobile device114 via download over network 109. That entity, for instance, might be amanufacturer of mobile device 114, a wireless service provider providingwireless services to mobile device 114, or a developer of applicationsfor mobile device 114. As with parts image processing system 104,application store 106 may have any number or combination of computingelements enabling it to communicate, store, and process data to carryout the disclosed techniques. In one embodiment, application store 106may store a mobile application (discussed below) that executes on mobiledevice 114 to enable user 112 to capture an image of a wear part ofmachine 110 and determine its degree of wear. User 112 may download theapplication to mobile device 114 from application store 106 over network109. In a “client-side” environment, however, parts image processingsystem 104 might offer the image-processing tasks as a service thatmobile device 114 accesses through a web browser or web applicationdownloaded from parts image processing system 104. In such aconfiguration, application store 106, and perhaps the mobileapplication, may be omitted.

Dealer system 108 may represent a computing system associated with adealer that sells or rents machine 110 or parts for machine 110. Thedealer may have a relationship with user 112, who may be a customer or apotential customer, and may have an interest in knowing the status ofmachine 110. For example, the dealer may desire to know when a wear partof machine 110 has become sufficiently worn so that it can inspect orservice machine 110 and potentially sell replacement parts or servicesto user 112. Like other elements of environment 100, dealer system 108may have any number or combination of computing elements enabling it tocommunicate, store, and process data to carry out the disclosedtechniques. Among other things, dealer system 108 may receivenotifications, such as e-mails or text messages, from other elements ofenvironment 100, such as mobile device 114 or parts image processingsystem 104, based on a determination that a wear part of machine 110 issufficiently worn. In response to such notifications, dealer system 108may initiate responses to address potential concerns with respect to aworn part. For example, dealer system 108 may contact user 112 to bringmachine 110 in for inspection or service, may arrange for a technicianof the dealer to visit machine site 102 and inspect or service machine110, or may prompt user 112 (via mobile device 114) to order areplacement for the worn part and allow user 112 to order the same.These and other characteristics of dealer system 108 are discussed indetail below.

Network 109 may represent any type or combination of electroniccommunication network(s) configured to communicate data between nodesconnected to network 109. For example, network 109 may represent theInternet, an Ethernet, a local area network (LAN), a wide area network(WAN), a personal area network (PAN), cellular network, a publicswitched telephone network (PSTN), or any combination thereof. In oneembodiment, network 109 may include a mobile network and relatedinfrastructure operable to provide Internet connectivity to mobiledevice 114, such as a 2^(nd) Generation (2G) cellular communicationnetwork, a 3^(rd) Generation (3G) cellular communication network, a3^(rd) Generation Long Term Evolution (LTE) network, or a 4^(th)Generation (4G) cellular communication network.

As mentioned, user 112 carries a mobile device 114 at machine site 102and uses it to capture a digital image of a wear part of machine 110,such as a link of tracks 116, which is used to determine its degree ofwear. To that end, mobile device 114 may embody any type of portablecomputing device equipped with a camera function and configured tocommunicate data over network 109. For instance, mobile device 114 maybe a smartphone, a cell phone, a tablet computer, a personal digitalassistant (PDA), a network-enabled digital camera, or other suchportable computing device.

FIG. 2 illustrates an exemplary embodiment of mobile device 114 indetail. As shown, mobile device 114 contains computing components thatenable it to transmit/receive, process, and store data. For example,mobile device 114 may have a processor 200, a main memory 202, aread-only memory (ROM) 204, an input device 206, an output device 208, awireless network communication interface 210, a camera 212, and astorage device 214, among other components.

Processor 200 may embody any general-purpose or special-purpose computermicroprocessor configured to execute computer program instructions,applications, or programs stored in main memory 202 and/or in storagedevice 214. Main memory 202 may include, for example, a random accessmemory (RAM) or other type of dynamic or volatile storage device. Mainmemory 202 may store information, instructions, programs, orapplications loaded from ROM 204 or storage device 214 for execution byprocessor 200.

ROM 204 may be any static or nonvolatile memory storage deviceconfigured to store computer program instructions, programs, orapplications for loading into main memory 202 and execution by processor200. For example, ROM 204 may be a programmable read-only memory (PROM),such as an erasable programmable read-only memory (EPROM), anelectronically erasable programmable read-only memory (EEPROM), aone-time programmable read-only memory (OTP NVM), a field-programmableread-only memory (FPROM), or a Flash memory device.

Input device 206 may embody one or more mechanisms that permit user 112to input information or commands to mobile device 114. For example,input device 206 may include a keyboard, a touch screen, a touch pad, amouse, a stylus, a voice-recognition device, a biometric-recognitiondevice, an accelerometer, a microphone, or any other type of device forenabling user input to a computing device.

Output device 208 may include one or more mechanisms that outputinformation to user 112 of mobile device 114. For example, output device208 may include a display device, a speaker, a vibrating device, alighting device or lamp, or any other type of device for providingoutput to a user of a computing device.

Wireless network communication interface 210 may include any device orsystem enabling mobile device 114 to communicate voice and/or data overelectronic communication network 109. For example, wireless networkcommunication interface 210 may include one or more radio antennas,transceivers, and/or other components for wireless communication. Suchwireless networks may include, for example, a cellular network, such asa 2^(nd) Generation (2G™), a 3^(rd) Generation (3G™), a 3^(rd)Generation Long Term Evolution (LTE™) network, or a 4^(th) Generation(4G™) network; a local area network (LAN), such as a Wi-Fi network(e.g., IEEE 802.11 protocol), or a WiMAX™ network (e.g., IEEE 802.16protocol); a picocell or femtocell network (e.g., a Bluetooth™ or otherunlicensed radio spectrum network); or other type of electroniccommunication network 109. Wireless network communication interface 210may include any components known in the art necessary to communicate onsuch network(s).

Camera 212 may embody any image-detection device mounted to or otherwiseassociated with mobile device 114 that captures an image within a viewof mobile device 114. For example, camera 212 may be a conventionalvisual-light-spectrum camera device mounted on mobile device 114 andoperable to capture and store a digital image in response to user 112providing appropriate input to input device 206, such as pressing a“soft” camera button displayed on a touch screen. Camera 212 may have anembedded image sensor (not shown) made up sensor pixels, such as acharge-coupled device (CCD). The sensor pixels may convert incidentelectromagnetic radiation focused thereon by a lens (not shown) intoelectrical charges for storage as a digital image. In other embodiments,camera 212 may be an infrared camera device or an X-ray camera device.Indeed, camera 212 may embody any type of device configured to captureelectromagnetic radiation as a digital image.

Consistent with the disclosed embodiments, user 112 may orient mobiledevice 114 so that a wear part (e.g., a link of tracks 116) is within afield of view of camera 212, and may provide input to input device 206to capture a digital image thereof. As explained below, mobile device114 and/or parts image processing system 104, depending upon theembodiment, may then process the digital image to determine the degreeof wear of the part.

Storage device 214 may include any type of mass data storage device onmobile device 114. For example, storage device 214 may embody asolid-state drive (SSD) or a magnetic data storage device, such as ahard drive. When user 112 captures a digital image with camera 212,processor 200 may store the image in storage device 214. Additionally,as shown in FIG. 2, in a “client-side” environment 100, storage device214 may store a mobile operating platform 216, as well as a wear partsapplication 218 and an associated wear parts library 220.

Mobile operating platform 216 may embody any type of software operatingenvironment for a mobile computing device in which one or more mobileapplications executes. For example, mobile operating platform 216 mayembody the Nokia Symbian™ operating environment, the Apple IOS™operating environment, the RIM Blackberry™ operating environment, theGoogle Android™ operating environment, the Windows Mobile™ operatingenvironment, or another graphical operating environment configured toexecute on a mobile computing device and support execution of mobileapplications.

Wear parts application 218 may embody an application configured toexecute on mobile operating platform 216 to perform functions fordetermining the degree of wear of wear parts of machine 110 based ondigital images captured with camera 212. In one embodiment, mobiledevice 114 may download wear parts application 218 (and wear partslibrary 220) from application store 106, over network 109. For example,user 112 may register for a wear parts service with dealer system 108and create an account for user 112 and/or one or more machines 110. Aspart of this process, user 112 may download and use wear partsapplication 218 and wear parts library 220 on a free basis, asubscription basis, or a one-time payment basis. Alternatively, user 112may download and use wear parts application 218 and wear parts library220 without registering or creating an account with the dealer.

Wear parts library 220 may contain wear parts information for a varietyof wear parts of a variety of different machines. In one embodiment, thewear parts information may be indexed by machine model, part name,and/or part model number, so that wear parts library 220 can be queriedto identify the wear parts information for a desired wear part. In oneembodiment, the wear parts information may include model information forthe wear parts. The model information may specify, for example, knownfeatures (e.g., dimensions and geometry) of wear parts, and may containinformation that enables mobile device 114 to identify the knownfeatures of the wear parts in a digital image thereof. The modelinformation may also identify wear edges of the wear parts and maycontain information enabling mobile device 114 to identify them indigital images of the wear parts. As used in this description, a “wearedge” refers to a surface of a wear part that is typically subject toforces during use, causing it to wear over time and eventually requiringthe part to be replaced. For example, as explained below, an engagementsurface of a tractor track link that engages the wheels of the tractorundercarriage is a wear edge.

The model information may also include dimension information and/ordesign specification information of the wear parts. Mobile device 114may use the dimension information and/or design specificationinformation, among other reasons, to orient digital images of wear partsso that accurate measurements can be taken from them, and to calibratemeasurements of wear parts made from digital images.

Additionally, wear parts library 220 may contain wear tables that enablemobile device 114 to determine the degree of wear of wear parts based onmeasurements of the wear parts taken from digital images thereof. In oneembodiment, the wear tables may define a degree of wear of a wear partas a function of a measured dimension of a wear edge of the wear part,such as a measured distance from a known feature of the wear part to thewear edge. As an example, a wear table for a track link may contain thefollowing information:

Track Link Wear Table Distance from Known Feature to Wear edge Degree ofWear 100 mm  0%  98 mm  10%  96 mm  20%  94 mm  30%  92 mm  40%  90 mm 50%  88 mm  60%  86 mm  70%  84 mm  80%  82 mm  90%  80 mm 100%As can be seen from the exemplary wear table, it defines a degree ofwear of the track link, in terms of a percentage, as a function of ameasured dimension of a wear edge of the track link—in this case, adistance from a known feature of the track link to a wear edge thereof.In this particular example, if the measured distance from the knownfeature to the wear edge of the track link is 94 mm, then the track linkis 30% worn, meaning that it has 70% of its useful life remaining beforeit should be replaced. Over time, as the track link wears, that distancemay eventually reduce to 80 mm, meaning that the track link has noremaining useful life and should be replaced immediately. It is notedthat a wear table need not take the form of a “table,” specifically. Forexample, a wear table could be an equation, formula, mathematical model,or other means for computing a degree of wear as a function of ameasured dimension of a wear edge of a wear part.

FIG. 3 is a representation of an exemplary track link 300 with certaincharacteristics highlighted to illustrate the application of the weartable and other information stored in wear parts library 220. As shownin the figure, track link 300 may have known features such as two pinsholes 302, 304 through which pins are inserted to connect track link 300to neighboring links in the track. Track link 300 may also have a wearedge 306 that engages a wheel 308 of the tractor as it travels. Overtime, as operation of the tractor creates friction forces between wheel308 and track link 300 that wear down wear edge 306, the distance 310between wear edge 306 and a line 312 connecting the centers of pin holes302, 304 decreases. That is, track link 300 wears as the perpendiculardistance 310 between the centers of pin holes 302, 304 and wear edge 306decreases. Pin holes 302, 304 are known features of track link 300 inthe sense that they remain in the same position throughout the life oftrack link 300. Accordingly, they may serve as a baseline against whichto measure the degree of wear of wear edge 306.

Known features of a wear part, such as pin holes 302, 304, may alsoserve as “landmarks” for orienting a digital image of the wear part. Tothis end, as explained above, the model information in wear partslibrary 220 may identify such known features of the wear parts and maycontain information enabling mobile device 114 to identify them indigital images of the wear parts. Thus, even if user 112 captures adigital image of a wear part that is rotated, skewed, off-center, etc.,the model information may be used to correct the orientation of theimage so that accurate measurements can be taken from it. For example,if user 112 captures a digital image of track link 300 that is rotated30 degrees clockwise from the model, processor 200 may identify thelocations of the centers of pin holes 302, 304 in the digital image,compare the identified locations of the centers of pin holes 302, 304 inthe image to their corresponding locations in the model for track link300 and, based on that comparison, rotate the image 30 degreescounterclockwise so that accurate measurements of track link 300 can bemade from the digital image. While use of the distance 314 between thecenters of pin holes 302, 304 is discussed above for calibratingmeasurements, it is also contemplated that other known features of awear part may be used for the same purpose. For example, a knowndiameter of a pin hole 302, 304 or other known features of the wear partmay be utilized to calibrate measurements taken from a digital image ofthe wear part.

Known features of a wear part, such as pin holes 302, 304, may alsoserve as bases for calibrating measurements taken from a digital imageof the wear part. For example, referring to FIG. 3, a distance 314between the centers of pin holes 302, 304 on a particular track link 300may be known (from the design specifications) to be 300 mm. To this end,as mentioned above, wear parts library 220 may contain dimensioninformation and/or design specification information for wear parts.Using this information, mobile device 114 may be configured to calibratedigital images of wear parts and/or measurements taken from them. In oneembodiment, mobile device 114 may be configured to analyze a digitalimage of a wear part to determine the distance, in pixels, between twoknown features of the wear part. Mobile device 114 may be configured todivide the distance between the two known features listed in the designspecifications by the determined pixel distance to obtain a calibrationfactor (distance per pixel) for the digital image. For example, ifmobile device 114 determines that there are 1000 pixels between thecenters of pin holes 302, 304 in a particular digital image of tracklink 300, and the pin holes 302, 304 are known to be 300 mm apart fromthe design specifications, mobile device 114 may calculate a calibrationfactor of 0.33 mm/pixel for that image. Subsequent pixel-distancemeasurements of track link 300 taken from that digital image may bemultiplied by the calibration factor to convert them to their distancein millimeters.

In a “client-side” environment, processor 200 may be configured toperform various functions relating to determining the degree of wear ofwear parts from digital images thereof. Generally, processor 200,executing wear parts application 218 on mobile operating platform 216,may be configured to orient a digital image of a wear part so that itcorresponds to a model of the wear part contained in wear parts library220 and accurate measurements can be taken from it. Processor 200,executing wear parts application 218 on mobile operating platform 216,may also be configured to calibrate the digital image of a wear part sothat pixel-based measurements of the wear part taken from the image canbe converted to real-world measurements (e.g., mm). Additionally,processor 200, executing wear parts application 218 on mobile operatingplatform 216, may be configured to analyze a digital image of a wearpart to measure a pixel distance from a known feature of the wear partto a wear edge of the wear part. Processor 200 may additionally beconfigured to convert the pixel distance to a real-world measurement(e.g., mm) using a calibration factor determined when calibrating theimage. Additionally, processor 200, executing wear parts application 218on mobile operating platform 216, may be configured to determine thedegree of wear of the wear part from the measurement of the distancebetween the known feature and the wear edge using a wear table stored inthe wear parts library 220. Processor 200, executing wear partsapplication 218 on mobile operating platform 216, may also be configuredto output an indication of the degree of wear to user 112 via outputdevice 208. These and other functions and configurations of processor200 are explained in greater detail below with respect to FIGS. 5-9.

As mentioned above, environment 100 may include parts image processingsystem 104 if a “server” or “cloud” configuration is desired. (But partsimage processing system 104 may be omitted, or its role limited, in a“client-side” environment.) If parts image processing system 104 isused, it may have similar components and functions as mobile device 114in a client-side environment. In this configuration, wear partsapplication 218 and wear parts library 220 may be omitted from mobiledevice 114, and their functionality replaced with correspondingcomponents and functionality on parts image processing system 104. Forexample, wear parts application 218 may provide a web-based applicationor site which mobile device 114 accesses over network 109 via a webbrowser. Using the web-based application or site, mobile device 114 maycapture and transmit a digital image of a wear part to parts imageprocessing system 104 over network 109. And parts image processingsystem 104 may process the digital image to determine the degree of wearof the wear part, send results to mobile device 114 over network 109 forpresentation to user 112, send a notification to dealer system 108, etc.

FIG. 4 illustrates a representation of components of parts imageprocessing system 104. As with other elements of environment 100, partsimage processing system 104 may have any number or combination ofcomputing elements enabling it to communicate, store, and process datato carry out the disclosed techniques. For example, parts imageprocessing system 104 may include one or more server computers having aprocessor 400, a main memory 402, a read-only memory (ROM) 404, anetwork communication interface 406 for communicating on network 109,and a storage device 408, which may have a similar makeup to thecomponents discussed above with respect to FIG. 2.

Within storage device 408, there may be stored a wear parts application410 and a wear parts library 412, which may have a similar makeup andperform similar functions as wear parts application 218 and wear partslibrary 220 of mobile device 114, discussed above. In this case,however, wear parts application 410 may be a web-based application orsite, which mobile device 114 accesses over network 109, instead of aclient-side application.

FIG. 5 is a representation of an exemplary wear parts applicationgraphical user interface (GUI) display 500, displayed on mobile device114. In a client-side environment, GUI display 500 may be displayed onoutput device 208 when user 112 launches wear parts application 218stored on mobile device 114 and processor 200 executes the same. In aserver or cloud environment, GUI display 500 may be displayed on outputdevice 208 when user 112 accesses a web-based wear parts application 410of parts image processing system 104 over network 109 using the mobiledevice's web browser.

As illustrated in the figure, GUI display 500 may include an imagecapture window 502 that displays a digital image of the field of view ofcamera 212. Holding mobile device 114, user 112 at machine site 102 mayorient mobile device 114 and camera 212 so that a wear part of machine110 (e.g., track link 300) is within the field of view of camera 212 anddisplayed in image capture window 502. In some embodiments, imagecapture window 502 may have a frame, outline, or other means (not shown)of illustrating to user 112 how to orient the wear part in image capturewindow 502 so that an appropriate digital image of the wear part can becaptured.

GUI display 500 may further include a capture image interface element504, such as a “soft” button. When user 112 selects capture imageinterface element 504, processor 200 may control camera 212 to capturethe digital image displayed in image capture window 502. Processor 200may also store the digital image in storage device 214.

GUI display 500 may include a find part interface element 506, which mayalso be a “soft” button, for example. When user 112 selects find partinterface element 506, processor 200 may access wear parts library 220and enable user 112 to search for and identify the particular wear partof which user 112 has or intends to capture an image. For example,processor 200 may prompt user 112 to enter the model of machine 110, thename of the wear part, or the model number of the wear part into inputdevice 206. Processor 200 may then query wear parts library 220 based onthe entered information to obtain a search results list of one or morecandidate wear parts, and may prompt user 112 to select a wear part fromthe list. User 112 may then identify and select the subject wear partfrom the list. Responsive to the selection, processor 200 may access therelevant model information, wear table(s), or other wear partsinformation stored in wear parts library 220 for the selected wear part.

GUI display 500 may also include a determine wear interface element 508,which may also be a “soft” button, for example. When user 112 selectsdetermine wear interface element 508, processor 200 may determine thedegree of wear of the wear part from the digital image. Briefly,responsive to selection of determine wear interface element 508,processor 200 may (optionally) orient the digital image of the wear partso that it corresponds to the accessed model of the wear part andaccurate measurements can be taken from it. Processor 200 may alsocalibrate the digital image of the wear part so that pixel-basedmeasurements of the wear part taken from the digital image can beconverted to real-world measurements (e.g., mm). Additionally, processor200 may analyze the digital image of the wear part to measure a pixeldistance from a known feature of the wear part to a wear edge of thewear part. Processor 200 may additionally convert the pixel distance toa real-world measurement (e.g., mm) using a calibration factordetermined when calibrating the image. Additionally, processor 200 maydetermine the degree of wear of the wear part from the measurement ofthe distance between the known feature and the wear edge using theaccessed wear table for the wear part. This description explains thesetasks in greater detail below with respect to the flowcharts of FIGS.6-9. In some embodiments, processor 200 may initially perform imageprocessing on the captured digital image to determine whether it issuitable for taking measurements of the wear part. If processor 200determines that the image is not suitable, processor 200 may prompt user112 to capture a new image.

As shown in FIG. 5, GUI display 500 may further include a wear indicatorinterface element 510 that indicates the degree of wear of the wearpart, as determined from the digital image. In one embodiment, wearindicator interface element 510 may be a meter, gauge, graph, or othergraphic that processor 200 animates to convey to the user the degree ofwear of the wear part. For example, wear indicator interface element 510may indicate the degree of wear as a percentage (0-100%), a color code(e.g., green-red), or a scale (e.g., 1-10). In one embodiment, ifprocessor 200 determines that the degree of wear is above a threshold(e.g., 70%), processor 200 may send a notification (e.g., a text messageor e-mail) to dealer system 108. The notification may contain, forexample, the identity of user 112, machine 110, the wear part, thedegree of wear, and/or other information that apprises the dealer of thesituation so that the dealer can take further action, if warranted.

FIG. 6 is a flowchart illustrating an exemplary method 600 fordetermining the degree of wear of a wear part from a digital image,consistent with the disclosed embodiments. In a “client-side”environment 100 in which mobile device 114 (rather than parts imageprocessing system 104) handles image processing, processor 200 of mobiledevice 114 may perform method 600 when executing wear parts application218 on mobile operating platform 216. It is noted that, depending uponthe desired implementation, steps of method 600 may be performed indifferent orders or even omitted entirely. The steps of method 600 areintended to illustrate potential embodiments consistent with thedisclosure rather than to limit the scope of the disclosure in any way.

In step 602, mobile device 114 may capture a digital image of a wearpart of machine 110. For example, user 112 at machine site 102 may havea concern that a track link 300 of machine 110 is worn, and may selectand launch wear parts application 218 on his or her mobile device 114.When GUI display 500 is up, user 112 may orient mobile device 114 suchthat track link 300 is within the field of view of camera 212 anddisplayed inside a frame of image capture window 502. User 112 may thencapture a digital image of track link 300 by selecting capture imageinterface element 504 of GUI display 500. In response, processor 200 maycontrol camera 212 to capture the digital image and store it in storagedevice 214 and/or in memory for processing.

In step 604, mobile device 114 may identify the wear part. For example,user 112 may select find part interface element 506 of GUI display 500.Processor 200 may display a prompt, such as a text entry field,drop-down menus, or other user interface element, allowing user 112 toprovide input to identify the particular wear part of which user 112 hascaptured an image—in this example, track link 300. User 112 may inputthe model of machine 110, the name of the wear part, the model number ofthe wear part, or other information identifying the wear part. Processor200 may then query wear parts library 220 based on the enteredinformation to identify wear part(s) that match the query. Processor 200may prompt user 112 to select a wear part from a list of searchresult(s) containing candidate wear part(s). User 112 may then identifyand select track link 300 from the list using input device 206.

In step 606, mobile device 114 may access information in wear partslibrary 220 associated with the selected wear part. For example,processor 200 may access in wear parts library 220 the modelinformation, dimension and design specification information, or weartable(s) for the selected wear part—in this example, track link 300.

In some embodiments, in step 608, mobile device 114 may orient thedigital image of the wear part captured in step 602 to match the modelinformation for the wear part, so that accurate measurements can betaken from it. Processor 200 may use any number of image-processingtechniques for orienting the image. For example, processor 200 may do soby analyzing the model information for the wear part to identify theposition/orientation of known features of the wear part in the model,analyzing the digital image to identify the position/orientation of theknown features of the wear part in the image, and comparing the two.Based on the comparison, processor 200 may determine a difference in theposition/orientation of points in the digital image relative to theposition/orientation of points in the model information. Processor 200may then change the position/orientation of the points in the digitalimage based on the difference so that they match theposition/orientation of the points in the model. This may involve, forexample, zooming, rotating, skewing, or shifting in the X, Y, or Zdirection the coordinates of points in the digital image so that theymatch the model.

For instance, in the example of the track link 300 wear part, processor200 may analyze the digital image to determine the position/orientationof pin holes 302, 304 in the digital image, determine theposition/orientation of pin holes 302, 304 specified in the modelinformation for track link 300, and compare the two to determine adifference in the position/orientation of the pin holes 302, 304 in thedigital image relative to their position/orientation in the model.Processor 200 may then change the coordinates of points in the digitalimage based on the difference so that their position/orientation is thesame as in the model information.

As mentioned above, in some embodiments, step 608 may be unnecessary andomitted. For example, method 600 may identify the wear part (step 604)before capturing the digital image of the wear part (step 602). In thisscenario, processor 200 may have already accessed the model informationfor the wear part in the wear parts library 220. And the modelinformation may include, for example, a frame or outline (not shown) ofthe wear part that defines the correct orientation that the wear partshould have when captured in a digital image. Processor 200 may displaythe frame or outline in image capture window 502 to illustrate thecorrect orientation to the user 112. With the visual aid of the frame oroutline, user 112 may orient mobile device 114 (and camera 212) withrespect to machine 110 such that the wear part displayed in the digitalimage is located within the frame or outline. In this manner, when user112 captures the digital image of the wear part, the image may alreadyhave the correct orientation indicated by the model information for thewear part and not require the reorientation of step 608.

In step 610, mobile device 114 may calibrate the captured digital imageso that accurate measurements can be made from it, as described above.FIG. 7 is a flowchart illustrating step 610 in detail. Generally,processor 200 may calibrate the digital image by measuring a dimensionof the wear part from the digital image and comparing the measureddimension to the known dimension of the wear part indicated by the modelinformation for the wear part.

As part of calibrating the digital image, in step 700, processor 200 mayidentify features of the wear part in the digital image. For instance,the model information for the wear part may indicate the generallocation (e.g., x-y coordinates) in which one would expect to findcertain features of the wear part in a correctly oriented digital image.And processor 200 may process the digital image to identify them anddetermine their precise location in the digital image. In the example ofa track link 300, processor 200 may process the digital image toidentify pin holes 302, 304, or the centers thereof. Depending on thetype of wear part, however, different types of wear part features may beidentified.

In step 702, processor 200 may measure a value of a dimension of thewear part based on the features of the wear part identified in thedigital image. For example, in one embodiment, processor 200 may measurea distance in pixels between the identified features. In the case oftrack link 300, for instance, processor 200 may determine the distance314 between the centers of pin holes 302, 304 by counting or otherwisedetermining the number of pixels between the centers of pin holes 302,304 in the digital image.

In step 704, processor 200 may determine a known value of the dimensionfrom the specification information for the wear part. For example, inthe case of track link 300, the model information accessed in step 606may contain specification information indicating that the actualdistance 314 between the centers of pin holes 302, 304 is 300 mm.

In step 706, processor 200 may calculate a calibration factor for thedigital image based on the measured value of the dimension of the wearpart and on the known value of the dimension of the wear part. In oneembodiment, processor 200 may divide the known value (e.g., units of mm)of the dimension of the wear part by the measured value (e.g., units ofpixels) of the dimension of the wear part to obtain the calibrationfactor (e.g., units of mm/pixel). For example, in the case of track link300, processor 200 may measure a distance 314 of 1,000 pixels betweenthe centers of pin holes 302, 304 in the digital image of track link300, and may determine from the specification information for track link300 that they are actually 300 mm apart. Based on these values,processor 200 may calculate a calibration factor of 0.333 mm/pixel(i.e., 300 mm/1000 pixels).

Returning to FIG. 6, in step 612, mobile device 114 may measure a wearedge of the wear part from the digital image to determine the degree ofwear of the wear part. FIG. 8 illustrates a flowchart showing step 612in detail. Generally, processor 200 may measure a wear edge of the wearpart from the digital image and compare that measurement to a wear tablefor the wear part to determine the degree of wear of the wear part.

In step 800, processor 200 may identify, in the digital image, a wearedge of the wear part. In one embodiment, the model information for thewear part may identify one or more wear edges of the wear part that canbe used to determine the degree of wear of the wear part. It may alsoindicate the general location (e.g., x-y coordinates) where one wouldexpect to find the wear edge(s) in a digital image oriented in the sameway as the model of the wear part. Using the model information,processor 200 may process the digital image to identify a wear edge anddetermine its precise position in the digital image. In the case oftrack link 300, for example, processor 200 may process the digital imageto identify wear edge 306 (that engages the tractor wheels 308 duringoperation) and determine its location in the digital image. In oneembodiment, processor 200 may define the wear edge as a line in thedigital image.

Alternatively or additionally to processor 200 processing the image toidentify the wear edge, wear parts application 218 may allow user 112 tovisually define the location of the wear edge on the displayed digitalimage. For example, user 112 may use input device 206 to draw or place aline in image capture window 502 that identifies the position of thewear edge in the digital image. In the case of track link 300, forexample, user 112 may input a line that is flush with wear edge 306.Processor 200 may then determine and store the position (e.g.,coordinates, angle, etc.) of the input line for use in determining thedegree of wear of the wear part.

In step 802, processor 200 may identify, in the digital image, one ormore features of the wear part that serve as a basis to measure the wearedge and determine the degree of wear of the part. Step 802 may besimilar to step 700 discussed above in connection with FIG. 7, and mayinvolve identifying the same or different features of the wear part. Inthe case of track link 300, for example, processor 200 may identify inthe digital image pin holes 302, 304, or the centers thereof, with theaid of model information for track link 300. (Depending upon the type ofwear part, however, different types of wear part features may beidentified.) In some cases, processor 200 may determine a line betweenthe identified features of the wear part and may use the line as a basisfor measuring the identified wear edge. In the case of track link 300,for instance, processor 200 may determine a line 312 between the centersof pin holes 302, 304 and use that line 312 as a basis for measuringwear edge 306, as explained above.

In step 804, processor 200 may measure, from the digital image, a valueof a dimension of the wear part based on the wear edge identified instep 800 and on the feature(s) of the wear part identified in step 802.In one embodiment, processor 200 may do so by measuring a distance inpixels between the identified wear edge and the identified feature(s) ora line between the identified feature(s). In the case of track link 300,for instance, the particular dimension to be measured may be thedistance 310 from a line 312 connecting the centers of pin holes 302,304 and the line, identified by processor 200 in the digital image orinput by user 112, defining wear edge 306. And processor 200 maydetermine the distance 310 by counting the number of pixels betweenthose two lines.

In step 806, processor 200 may convert the measured value of thedimension to a “real-world” measurement based on the calibration factorcalculated in step 706. In one embodiment, processor 200 may do so bymultiplying the measured value by the calibration factor. Continuingwith the track link example, processor 200 may have determined adistance 310 of 282 pixels between line 312 connecting the centers ofpin holes 302, 304 and the line defining wear edge 306 and a calibrationfactor of 0.333 mm/pixel. Processor 200 may multiply 282 pixels X 0.333mm/pixel to obtain a distance 310 of about 94 mm between line 312connecting the centers of pin holes 302, 304 and the line defining wearedge 306.

In step 808, processor 200 may determine the degree of wear of the wearpart using the converted value determined in step 806. In oneembodiment, processor 200 may look up the converted value in the weartable for the wear part to determine the degree of wear. For instance,in the case of track link 300, having determined a distance 310 of 94 mmbetween line 312 connecting the centers of pin holes 302, 304 and theline defining wear edge 306, processor 200 may look up 94 mm in the weartable (shown above) for track link 300 to determine that track link 300is 30% worn.

Returning to FIG. 6, in step 614, mobile device 114 may display anindication of the degree of wear of the wear part determined in step808. For example, processor 200 may animate wear indicator interfaceelement 510 to illustrate the degree of wear. Continuing with the tracklink example, processor 200 may animate wear indicator interface element510 to indicate that track link 300 is 30% worn, such as by illuminatingthree out of ten total bars of wear indicator interface element 510.

In step 616, mobile device 114 may optionally determine whether thedegree of wear of the wear part is greater than a threshold, such as70%. If the degree of wear is greater than the threshold, mobile device114 may notify dealer system 108 (step 618), such as by sending a textmessage, e-mail, or other electronic message over network 109. Asdiscussed above, the message may contain information about machine 110,the wear part, user 112, machine site 102, etc., that enables the dealerto assess the situation and take further action if warranted. Responsiveto the message, dealer system 108 may contact user 112 to bring machine110 to the dealer for inspection or service, may arrange for atechnician to visit machine site 102 and inspect or service machine 110,or may prompt user 112 to order a replacement for the wear part andallow user 112 to order the same. Alternatively or additionally tonotifying the dealer, mobile device 114 may display a warning, vibrate,sound an alert, or otherwise draw the attention of user 112 if thedegree of wear is greater than a threshold.

Method 600 may end after completion of step 618 or after completion ofstep 616, in the event that mobile device 114 determines that the degreeof wear is not greater than the threshold.

FIG. 9 is a flowchart illustrating an exemplary method 900 fordetermining the degree of wear of a wear part from a digital image, in a“cloud” or “server” embodiment in which parts image processing system104 handles image processing. The embodiment of FIG. 9 is similar to theembodiment of FIG. 6 except that processor 400 executing wear partsapplication 410 of parts image processing system 104 may performfunctions or steps of method 900 that are handled by mobile device 114in the embodiment of FIG. 6.

User 112 may use a browser of mobile device 114 to access, over network109, a web page or web application associated with the parts imageprocessing system 104's wear parts application 410. In step 902, user112 may use the web page or application to capture a digital image of awear part in a similar manner as described above in step 602. In step904, user 112 may also use the web page or application to identify thewear part captured in the image, in a similar manner as described abovein connection with step 604. In step 906, user 112 may use the web pageor application to send the captured digital image and the identificationof the wear part from mobile device 114, over network 109, to partsimage processing system 104. Accordingly, as part of step 906, partsimage processing system 104 may receive the captured image and theidentification of the wear part from mobile device 114 over network 109.

In step 908, parts image processing system 104 may access modelinformation contained in wear parts library 412 for the identified wearpart, in a similar manner as described above in connection with step606. In step 910, parts image processing system 104 may optionallyorient the digital image of the wear part captured in step 902 to matchthe model information for the wear part, so that accurate measurementscan be taken from it, in a similar manner as described above inconnection with step 608. In step 912, parts image processing system 104may additionally calibrate the digital image to determine a calibrationfactor for the digital image, in a similar manner as described abovewith respect to step 610. Additionally, in step 914, parts imageprocessing system 104 may measure a wear edge of the wear part from thedigital image, in a similar manner as described above with respect tostep 612.

In step 916, parts image processing system 104 may send an indication ofthe determined degree of wear to mobile device 114 over network 109.Accordingly, as part of step 916, mobile device 114 may receive theindication over network 109. Additionally, in step 918, parts imageprocessing system 104 may display the indication to user 112 asdescribed above with respect to step 614. Additionally, in step 920,parts image processing system 104 may determine whether the degree ofwear of the wear part is above a threshold, as described above withrespect to step 616, and may notify the dealer (step 922) if so.

FIG. 10 is an alternate representation of a wear part, in this case,track link 1000, with certain characteristics highlighted to illustratean alternate technique for determining wear. As shown in the figure,track link 1000 may have known features such as a pin hole center 1002,chamfer edge 1004, pin-link boundary 1006, cast surface boundary 1008,and wear edge 1010.

The distances between pin hole center 1002 and chamfer edge 1004,pin-link boundary 1006, cast surface boundary 1008, and wear edge 1010may be stored in wear parts library 220 FIG. 2). Additionally, thedistance from the pin hole center 1002 to wear edge 1010 when track link1000 is determined to have reached 100% wear, and/or various otherpercentages of wear, may also be stored in wear parts library 220 (FIG.2). The concentric circles composed of chamfer edge 1004, pin-linkboundary 1006, and cast surface boundary 1008 may be applied within wearparts application 218 running on mobile device 114 to locateautomatically pin center 1002 and calibrate the distance per pixelbetween the pin center 1002 and wear edge 1010 (and thereby determinethe degree of wear).

FIG. 11 is a flowchart describing an alternative method 1100 ofdetermining the degree of wear of a wear part from a digital image,consistent with the disclosed embodiments. Like method 600 in a“client-side” environment 100 in which mobile device 114 (rather thanparts image processing system 104) handles image processing, processor200 of mobile device 114 may perform method 1100 when executing wearparts application 218 on mobile operating platform 216. It is notedthat, depending upon the desired implementation, steps of method 1100may be performed in different orders or even omitted entirely. Aspectsof method 1100 may also be performed in a “cloud” or “server”environment, similar to method 900, with parts image processing system104 handling image processing and/or other steps of method 1100. Thesteps of method 1100 are intended to illustrate potential embodimentsconsistent with the disclosure rather than to limit the scope of thedisclosure in any way.

In step 1102, a user of mobile device 114 identifies the wear part. Forexample, the user of mobile device 114 may input the part number and/orthe machine serial number via wear parts application 218 using inputdevice 206.

In step 1104, wear parts library 220 may be accessed to retrieve datafor the wear part, the image capture process to be used for the wearpart, and/or user instructions to properly capture a digital image ofthe wear part. For example, the dimensions of wear part discussed abovein connection with FIG. 10 may be retrieved. Based on thecharacteristics of the wear part (e.g., size), specific image captureinstructions may be stored in wear parts library 220. For example,smaller wear parts may only require a single image to capture all theinformation necessary to make the wear determination, while larger partsmay require multiple images stitched together. The information retrievedin step 1104 may include a set of instructions and image overlays forthe user. The instructions and image overlays direct the user how toposition mobile device 114 and/or camera 212 in relation to the wearpart to capture one or more digital images.

In step 1106, the user may be instructed to position mobile device 114and/or camera 212 to capture an image of the wear part centered over pinhole center 1002. In one embodiment, mobile device 114, based on theinformation retrieved in step 1104, may display an overlay (e.g., aframe, outline, or crosshair) illustrating how to orient the wear partwithin image capture window 502 over pin hole center 1002.

In step 1108, once the user has positioned mobile device 114 and/orcamera 212, the user may provide input to input device 206 (e.g., tap acapture image interface element 504 on the touchscreen) to initiatecapturing a first digital image of the wear part, which will bediscussed below in connection with FIG. 12.

In step 1110, using similar techniques as in step 1106, the user may beinstructed to position mobile device 114 and/or camera 212 to capture animage of pin hole center 1002 and wear edge 1010. In one embodiment, theuser may be instructed via an overlay (e.g., frame, outline, orcrosshair) in image capture window 502 to position both pin hole center1102 and wear edge 1010 within the view. For example, an overlaid crosshair may identify the desired position of pin hole center 1002, and anoverlaid line may identify the desired position of wear edge 1010.

In step 1112, once the user has positioned mobile device 114 and/orcamera 212 as instructed, the user may provide input to input device 206(e.g., tap a capture image interface element 504 on the touchscreen) toinitiate capturing a second digital image of the wear part, as will bediscussed below in connection with FIG. 12.

In step 1114, using similar techniques as in steps 1106 and 1110, theuser may be instructed to position mobile device 114 and/or camera 212to capture an image of wear edge 1010. In one embodiment, the user maybe directed to position mobile device 114 and/or camera 212 in order toalign an image overlay (e.g., a frame, outline, or crosshair) with wearedge 1010 within image capture window 502. Additionally, the user may beinstructed to position mobile device 114 and/or camera 212 closer to orfarther from the wear part so that wear edge 1010 is in the center ofimage capture window 502 and at least a portion of the circular arc(s)defined by chamfer edge 1104, pin-link boundary 1106, and/or castsurface boundary 1108 are in the image. This adjustment of focaldistance may be used to accomplish the image stitching in step 1120,discussed below.

In step 1116, once the user has positioned mobile device 114 and/orcamera 212 as instructed, the user may provide input to input device 206(e.g., tap a capture image interface element 504 on the touchscreen) toinitiate capturing a third digital image of the wear part, as will bediscussed below in connection with FIG. 12.

In step 1118, the first, second, and third images captured in thepreceding steps may be processed. Various types of image processingknown in the art may be performed to prepare the images for the weardetermination. For example, wear parts application 218 may correct theimages for lens distortion. The distortion correction may beaccomplished by techniques known in the art, such as through a one-timecalibration or a correction based on an algebraic lens distortion model.

In step 1120, once the images have been processed, wear partsapplication 218 may stitch the images together. The stitching may bedone according to various techniques known in the art, such as byapplying known algorithms to leverage invariant features among theimages.

In step 1122, the known features of the wear part may be located in theimages. For example, pin hole center 1002, wear edge 1010, and any ofchamfer edge 1004, pin-link boundary 1006, or cast surface boundary 1008may be located in the stitched digital images. Because the user wasinstructed to position mobile device 114 and/or camera 212 in a specificmanner for the different images, wear parts application 218 may use thatinformation identify one or more of these features. For example, wearparts application 218 may apply an arc-origin-finding algorithm toidentify pin hole center 1002; a polar plot edge detection algorithm toidentify any of chamfer edge 1004, pin-link boundary 1006, or castsurface boundary 1008; and standard edge detection to identify wear edge1010. As a result of step 1122, location information for pin hole center1002, wear edge 1010, and any of chamfer edge 1004, pin-link boundary1006, or cast surface boundary 1008 may be determined.

In step 1124, using the location information derived in step 1122 alongwith information in wear parts library 220, wear parts application 218may calculate a distance from pin hole center 1002 to wear edge 1010. Inone embodiment, the ratio of the pixel distance and/or physical distancebetween pin hole center 1002 to the arc boundary of any of chamfer edge1004, pin-link boundary 1006, or cast surface boundary 1008 may bedetermined. That ratio may then be compared to the pixel distance frompin hole center 1002 to wear edge 1010 to calculate the physicaldistance from pin hole center 1002 to wear edge 1010.

In step 1126, the distance from pin hole center 1002 to wear edge 1010may be compared to the wear table for the wear part stored in wear partslibrary 220. Based on the comparison, the percentage or degree of wearof the wear part may be determined, as described above. The percentageor degree of wear may then be displayed on mobile device 114 in step1128, as discussed above in connection with FIG. 5.

In some embodiments, the results of step 1126 may be sent to dealersystem 108 over network 109 using mobile device 114's wireless networkcommunication interface 210. For example, the wear percentage or degree,part number, part serial number, and/or machine serial number may betransferred. Additionally, mobile device 114 may transfer the capturedimages, the time required to complete the wear determination, and imageprocessing performance information for steps 1108, 1112, and 1116-1122.

FIG. 12 is a representation of image capturing steps 1108, 1112, and1116 in greater detail. These steps may include various assessments toconfirm whether the image (currently in the image capture window 502 ofmobile device 114) is suitable for the wear determination and, thus,should be captured. In one embodiment, an exposure quality assessmentprocess 1202, a focus quality assessment process 1204, an accelerationassessment process 1206, and an orientation/position assessment process1208 may be performed. They may be performed simultaneously or in asequence. It is to be appreciated, however, that additional, fewer,and/or different image assessments may be made as part of image capture,if desired. In one embodiment, each assessment process 1202-1208 mayoutput a value indicating whether it has assessed the image as suitablefor the wear part determination (e.g., yes or no, 0 or 1, or a valuebetween 1 and 10)

Exposure quality assessment process 1202 may assess the exposure qualityof the image and report a value indicating whether the image exposure issuitable for the wear determination. In one embodiment, it may analyzethe distribution pixel darkness in the image to determine whether theimage is underexposed or overexposed and output a value indicating thesame (e.g., yes or no, 0 or 1, or a value between 1 and 10). If theimage is underexposed, exposure quality assessment process 1202 mayalternatively report a recommendation to increase the f-stop or turn onthe flash of camera 212. If the image is overexposed, it may send arecommendation to reduce the f-stop, turn off the flash, or shield thelens.

Focus quality assessment process 1204 may check whether mobile device114's processor indicates that lens focus has been obtained. Once lensfocus is obtained, focus quality assessment process 1204 may optionallyassess focus based on image frequency entropy in a sample region of theimage or by other means. In any event, focus quality assessment process1204 may output a value indicating whether proper focus has beenobtained for the wear determination (e.g., yes or no, 0 or 1, or a valuebetween 1 and 10).

If mobile device 114 contains an accelerometer, acceleration assessmentprocess 1206 may determine whether the mobile device 114's currentacceleration might inhibit capturing an image that is suitable for thewear determination. In one embodiment, it may report the currentacceleration of mobile device 114 along with a value indicating whetherit expects the image to be of suitable quality for making a weardetermination based on the amount of acceleration (e.g., yes or no, 0 or1, or a value between 1 and 10).

Orientation/position assessment process 1208 may determine theorientation of the image and output a value indicating whether the imageis properly oriented for performing the wear determination. For example,orientation/position assessment process 1208 may determine that the useris nominally aligning camera 212 to the wear part in prescribed relativepositions and orientations. In step 1106, for example,orientation/position assessment process 1208 may capture intermediateimages and measure the elliptical distortion in the arc boundary of anyof chamfer edge 1004, pin-link boundary 1006, or cast surface boundary1008. In one embodiment, orientation/position assessment process 1208may apply a specialized algorithm that reduces the image processingrequired to identify the pin hole center 1002 and the edgerepresentation of the arc(s). Then, it may apply standard ellipticalmeasurement techniques to output its orientation/position assessment(e.g., yes or no, 0 or 1, or a value between 1 and 10).

Orientation/position assessment process 1208 may quickly assess theplacement of mobile device 114 and/or camera 212 with respect to wearedge 1010. For some wear parts, the wear edge 1010 desired to becaptured in the image may be closer to the lens of camera 212 thanbackground objects adjacent the wear edge 1010 that are also in theimage. This may cause variations in hue, saturation, and brightness oneither side of the wear edge 1010 in the image. Accordingly, in oneembodiment, orientation/position assessment process 1208 may average thehue, saturation, and/or brightness values on both sides of wear edge1010 and in strips parallel to the expected wear edge 1010. Andorientation/position assessment process 1208 may assess that the imageis properly oriented/positioned based on an agreement of the averagedvalues on either side of the expected wear edge 1010.

An image optimization controller process 1210 may receive the assessmentvalues output by assessment processes 1202-1208. In one embodiment,image optimization controller process 1210 may weigh and track thevalues over time, adjust control settings, and/or give directions to theuser to alter the conditions under which the image is being captured.Image optimization controller 1210 may analyze the weighted assessmentsover time to determine when the picture is optimized (step 1212) (i.e.,suitable for the wear determination) and then capture the image (step1214). Of course, as reflected in FIG. 12, the assessment processes1202-1208 may continue until image optimization process 1210 determinesthat the image is optimized and suitable for the wear determination.

In one embodiment, processes 1202-1214 may be implemented by mobiledevice 114's processor 200 executing software modules of wear partsapplication 218 in a “client-side” embodiment. In “cloud” or “server”environments, the captured image(s) may be sent to parts imageprocessing system 104, and processor 400 executing software modules ofwear parts application 410 may perform processes 1202-1212 and thentransmit an indication to mobile device 114 whether the captured imageis suitable for the wear determination or whether the user shouldcapture another image.

INDUSTRIAL APPLICATION

The disclosed systems and methods find application in any environment inwhich a user wishes to determine the degree of wear of a wear part. Byusing a mobile device to capture a digital image of the wear part anddetermine the degree of wear of the part from the digital image, thedisclosed systems and methods allow the user to easily assess the partwithout necessarily having detailed knowledge about the part, its wearcharacteristics, or the machine.

For example, a user 112 of environment 100 may operate a tractor at aworksite. Upon traversing steep terrain, the tracks 116 of the tractormay slip from the teeth that engage the tracks to the tractor's wheels308 (FIG. 3). Hearing the noise, the operator may stop the tractor,dismount, and examine the track that slipped. Finding a track link 300that appears damaged, the operator may pull out his or her mobile device114 and launch a wear parts application 218 (FIG. 2) installed on it.The operator may orient the mobile device 114 and its camera 212 (FIG.2) so that the track link 300 is displayed inside an image capturewindow 502 (FIG. 5) of the wear parts application 218. The operator mayproceed to capture a digital image of the track link 300 by selecting acapture image interface element 504 of the application, and may identifythe track link 300 to the application via find part interface element506. The operator may then select a determine wear interface element 508of the application, in response to which mobile device 114 may determinefrom the digital image that the track link 300 is 80% worn and displayan indication of the same via a wear indictor interface element 510 ofthe application. At the same time, mobile device 114 might also send amessage to dealer system 108 over the network 109 so that the dealer canschedule a service appointment with the operator to replace the tracklink 300.

The disclosed techniques have additional application as well. Forexample, mobile device 114 and/or parts image processing system 104 maydetermine a degree of “sag” of tracks 116 from a digital image thereof.As tracks 116 wear, they may slowly sag over time in the center (betweenthe wheels) from an initial height when they are new. Accordingly, amodel and/or wear table for tracks 116 (when new) may be contained inwear parts library 218 and/or 412. The model and/or wear table maydefine an amount of sag for tracks 116 as a function of a degree of wearof the tracks 116, e.g., from 0% worn (new) to 100% worn. For example,the model and/or wear table may define the sag as a measured distancefrom a specified initial height of the center of tracks 116 (when theyare new) to a measured (i.e., current) height of the center of tracks116 as measured from the digital image, that is, a change in height oftracks 16. In some embodiments, a known feature of tracks 116 or machine116 may serve as a reference point to measure the height of tracks 116from the digital image. As an example, a sag of 10 mm may correspond toa 20% wear. Environment 100 may perform many of the same functionsdescribed above in connection with determining the degree of sag oftracks 116.

One of ordinary skill in the art will appreciate that computer programsfor implementing the disclosed techniques may be stored on and/or readfrom computer-readable storage media. The computer-readable storagemedia may have stored thereon computer-executable instructions which,when executed by a computer processor, cause the computer to perform,among other things, processes disclosed herein. Exemplarycomputer-readable storage media may include magnetic storage devices,such as a hard disk, a floppy disk, magnetic tape, or another magneticstorage device known in the art; optical storage devices, such asCD-ROM, DVD-ROM, or another optical storage device known in the art;and/or electronic storage devices, such as EPROM, a flash drive, oranother integrated circuit storage device known in the art. Thecomputer-readable storage media may be embodied by or in one or morecomponents of environment 100 (FIG. 1).

While illustrative embodiments of the invention have been describedherein, the scope of the invention includes any and all embodimentshaving equivalent elements, modifications, omissions, combinations(e.g., of aspects across various embodiments), adaptations, and/oralterations as would be appreciated by those in the art based on thepresent disclosures. The limitations in the claims are to be interpretedbroadly based on the language employed in the claims and not limited byexamples described in the present specification or during theprosecution of the application. Further, the steps or processes of thedisclosure may be modified in any manner, including by reordering steps,inserting steps, deleting steps, or having steps or functions performedby other entities of environment 100, without departing from theprinciples of the disclosure.

Additionally, for example, known dimensions of features of a wear partmay be used to calibrate a digital image. For example, in the case of atrack link 300, pin holes 302, 304 may have known diameters, a pin plugmay have a known diameter, a pin center chamfer may have a knowndiameter, a link bore may have a known diameter, and/or a link spotfacemay have a known diameter. Any of these known diameters or otherdimensions of a wear part may be compared to pixel-based measurements ofthe same, taken from a digital image of the part, to obtain theabove-described calibration factor.

It is intended, therefore, that the specification and examples beconsidered as exemplary only, with a true scope and spirit of thedisclosure being indicated by the following claims and their full scopeof equivalents.

What is claimed is:
 1. A method for determining part wear using a mobiledevice, comprising: capturing, using the mobile device, at least onedigital image of a wear part of a machine; identifying, using the mobiledevice, in the at least one digital image, a wear edge of the wear part;identifying, using the mobile device, in the at least one digital image,at least one feature of the wear part; measuring, using the mobiledevice, in the at least one digital image, the wear edge of the wearpart, including measuring, in the at least one digital image, a value ofa dimension of the wear part based on the identified wear edge and onthe identified at least one feature, wherein measuring a value of adimension of the wear part includes measuring a distance from a lineconnected with the identified at least one feature and a line definingthe identified wear edge in the at least one digital image; anddetermining, using the mobile device, the degree of wear of the wearpart based upon the measurement of the wear edge.
 2. The method of claim1, further comprising executing, by the mobile device, an application tocapture the at least one digital image and to determine the degree ofwear of the wear part.
 3. The method of claim 1, wherein measuring avalue of a dimension of the wear part includes measuring a distance fromthe identified at least one feature to the identified wear edge in theat least one digital image.
 4. The method of claim 3, wherein measuringa distance includes determining a number of pixels between theidentified at least one feature and the identified wear edge in the atleast one digital image.
 5. The method of claim 1, wherein determining adegree of wear of the wear part includes looking up the measured valueof the dimension of the wear part in a wear table that defines thedegree of wear of the wear part as a function of the value of thedimension.
 6. The method of claim 1, wherein determining a degree ofwear of the wear part further includes: calibrating the at least onedigital image; and converting the measurement based on the calibration.7. The method of claim 6, wherein calibrating the at least one digitalimage includes: identifying, in the at least one digital image, a firstfeature and a second feature of the wear part; measuring, in the atleast one digital image, a distance between the first feature and thesecond feature; determining a known distance between the first featureand the second feature based on specification information for the wearpart; and calculating a calibration factor based on the measureddistance and the known distance, and wherein converting the measurementbased on the calibration includes converting the measurement based onthe calibration factor.
 8. The method of claim 1, further includingdisplaying, by the mobile device, an indication of the determined degreeof wear of the wear part.
 9. The method of claim 1, further including:determining, by the mobile device, whether the degree of wear is greaterthan a threshold; and sending, by the mobile device and based upon adetermination that the degree of wear is greater than the threshold, anotification to a dealer system associated with the machine.